In the art of signal processing, particularly where the signal of interest is representative of some form of information--e.g., data, text, images, voice, video, etc., it will often be desirable to encode an input signal as a means for achieving a more efficient expression of that input signal. Such an encoding of an input signal is typically carried out by evaluating the input signal relative to a codebook established to encompass a given class of such signals.
Such a codebook may be established by operating on a large database of signals--selected from the general class of signals to be encoded--to find a small set of signals which can be used, often in a combinatorial arrangement, to represent, or encode, input signals of that general class. That small set of signals will then comprise the codebook. The operation of finding the codebook is sometimes characterized as "learning".
Two widely used learning algorithms are vector quantization (VQ) and principal component analysis (PCA), which are based on two fundamentally different ways of encoding data. In VQ, an input signal is encoded as the index of the closest codebook vector stored in memory. With PCA, on the other hand, an input signal is encoded as the coefficients of a linear superposition of a set of codebook vectors. However, neither PCA nor VQ, is able to learn a codebook consisting of vectors that are interpretable as features.